So welcome to today's AI2 lecture.

I'm filling in for Michael Kohlhaase.

We're in the last two weeks of the semester.

There's two big sections on natural language processing.

Natural language processing has recently been in the news a lot, so I'm assuming most of

you, or maybe all of you, already have some idea about it.

That's because of things like Google Translate and chat GPT, like modern AI systems that

heavily use the methods that we've learned in the lecture at large, large scales to reach

of proficiency that often mirror human intelligence.

We're going to look at this in two different chapters.

The current one, Chapter 12, is more about symbolic AI methods, and then Chapter 13 is

more about the sub-symbolic methods.

The difference between the two is sub-symbolic is where you have no idea what's going on

because the neural network is doing something and you don't really know why.

So it's really just one big soup of data.

Throw it into the network and be amazed at what comes out.

Chapter 12, the symbolic part, is where we try to actually understand what's happening,

and that usually means entirely different methods.

And what humans do is they somehow combine the two.

So if you think about understanding a sentence, you would normally have an immediate understanding

of what it means without really thinking about it, but you also know how the sentence has

a syntax tree based on the grammar of the language.

When you're learning a new language, you spend a lot of time learning the grammar.

Words are not really a list of words, but a tree of words, and the words have endings

and so on that they have to match.

There are all those kinds of rules that you memorize, and you also learn the meaning of

all the words.

So you spend a lot of time really learning the rules, and at some point, somehow your

brain starts doing all this stuff automatically, and you don't really think about the rules

anymore.

And the sub-symbolic AI methods try to model the second part, and it is currently not understood

whether we need both at the same time or if we can somehow be successful with just one

of them.

I mean, the purely symbolic methods have been unsuccessful for decades.

They've done some good things, but they haven't really reached the scale that we have now

with large language models.

But then we all know that the large language models also aren't really working.

They're basically just guessing.

It's like when you're talking to someone who knows all the language in general, who has

some background knowledge, and then they just start rambling.

And depending on how smart the person is, they might get it right most of the time,

but you never know if they're making stuff up.

Because the neural network really doesn't understand the roots.

The roots never went into the large language model at all.

They went into it implicitly because the training data conformed to the rules.

So it's basically an exercise in learning a language from scratch if you just listen

to people.

Human babies can do that.

Human adults usually not.

So it's really hard to understand if this will work or not.